Operator Reverse Monotonicity of the Inverse

I learned econometrics from Fumio Hayashi in 2006 using his textbook Econometrics. In an exercise related to deriving the variance matrix of efficient GMM on p. 245, Hayashi states without proof

Let \(A\) and \(B\) be two positive definite and symmetric matrices. \(A-B\) is positive semidefinite if and only if \(B^{-1}-A^{-1}\) is positive semidefinite.

I am a mathematically oriented person who is skeptical about mathematical statements until seeing their proofs. Although I found the proof in Horn & Johnson (1985) Matrix Analysis, Corollary 7.7.4(a), their proof was not very intuitive. In 2010, I came up with a very simple proof based on convex conjugate functions that work for any self-adjoint linear operators on Hilbert spaces.